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Abstract —We present a controller synthesis algorithm for 
reach-avoid problems for piecewise linear discrete-time systems. 
Our algorithm relies on SMT solvers and in this paper we 
focus on piecewise constant control strategies. Our algorithm 
generates feedback control laws together with inductive proofs 
of unbounded time safety and progress properties with respect 
to the reach-avoid sets. Under a reasonable robustness assump¬ 
tion, the algorithm is shown to be complete. That is, it either 
generates a controller of the above type along with a proof of 
correctness, or it establishes the impossibility of the existence 
of such controllers. To achieve this, the algorithm iteratively 
attempts to solve a weakened and strengthened versions of 
the SMT encoding of the reach-avoid problem. We present 
preliminary experimental results on applying this algorithm 
based on a prototype implementation. 

I. Introduction 

A Satisfiability Modulo Theory (SMT) problem is a clas¬ 
sical decision problem in computer science [6]. It takes as 
input a logical formula in first-order logic that can involve 
combinations of background theories, and requires one to 
decide whether or not the formula has a satisfying solution. 
For a bounded time horizon k, a simplest SMT problem 
in Equation ([T]i, for instance, is an encoding of a search 
for a sequence of control inputs vectors ui,... ,Uk that 
drives a discrete time linear open-loop control system from 
every initial state in the hypercube [0,0.1]” to the hypercube 
[0.9,1]" in k steps, while always keeping the state inside the 
hypercube [0,1]". 

3 ui,...Mfc,V a;o e [0,0.1]”,V t e {1,... ,k - 1}, 

let Xt+i = Axt + But 

such that Xt e [0,1]" and x^ e [0.9,1]”. (1) 

This example has several constraints that are defined in 
terms of the quantified variables Ui and Xi, numerical 
constants (including those in the matrices A and B), and 
the background theory of linear real arithmetic. An SMT 
solver is a software tool that solves SMT problems by 
either giving an assignment to the variables that satisfy 
all the constraints or by saying that none exists. Modern 
SMT solvers routinely handle linear problems with thousands 
of variables and millions of constraints, so much as they 
have become the engines for innovation in verification and 
synthesis for computer software and hardware [2], [10], [13]. 
Although many control systems can only be modeled by 
means of nonlinear arithmetic over the real numbers in¬ 
volving transcendental functions that make the corresponding 
SMT problems undecidable, the solvers are evolving rapidly 
and several incorporate approximate decision procedures for 
nonlinear arithmetic [15]. These technological developments 


motivated us (and others [20], [21]) to explore SMT-based 
controller synthesis. 

In this paper, we present an algorithm that uses SMT 
solvers for synthesizing controllers for discrete time systems. 
The dynamics of the system is given as a piecewise linear 
feedback control system. The control requirements are the 
standard reach-avoid specification [7], [8]: a set of states 
Goal that has to be reached while always staying inside a 
Safe set. 

A key difficulty in using SMT for synthesis, is that the 
resulting SMT problem has to encode the unrolled dynamics 
of the system with the unknown controller inputs. In the 
above simple example, this gave rise to k control input vari¬ 
ables and the intermediate states. For more general nonlinear 
models, the intermediate states cannot be written down in 
closed form and one has to unroll the over-approximations 
of the dynamics. This can then lead to overtly conservative 
answers from the solver. We present a technique that avoids 
this problem by synthesizing the control law together with 
an inductive proof of its correctness. The proof has two 
parts: (a) an inductive invariant that implies safety and (b) a 
ranking function that implies progress. A positive side-effect 
of this is that it can not only synthesize controllers with 
understandable correctness proofs, but it can also establish 
the nonexistence of provably correct controllers (of a certain 
template). 

In Section III we define the system model, the reach-avoid 
synthesis problem and a particular notion of robustness of 
models. In Section|IV|we first present a basic SMT encoding 
of the synthesis problem and then a strengthened and a weak¬ 
ened version this encoding. Using these two encodings, in 
Section |V] we present the synthesis algorithm, its soundness 
and relative completeness. In Section we illustrate an 
application of the algorithm in a vehicle navigation problem 
and conclude in Section Ivnl 


II. Related Work 

Researchers have recently used SMT solvers for synthesiz¬ 
ing programs and strategies in games. The approach in [21] 
uses SMT to find controllers for general linear temporal logic 
(LTL) specifications by stitching together motion primitives 
from a library. Unlike our encoding with inductive proofs, the 
approach of [21] involves bounded unrolling of the dynamics. 

In [20], the authors used SMT solvers to synthesize 
integrated task and motion plans by constructing a placement 
graph. In [4], a constraint-based approach was developed 
to solve games on infinite graphs between the system and 



the adversary. Our work can be seen as introducing control 
theoretic constraints to the SMT formulation. 

The authors of [11], [26] proposed a game theoretical 
approach to synthesize controller for the reach-avoid prob¬ 
lem, first for continuous and later for switched systems. In 
these approaches, the reach set of the system is computed 
by solving a non-linear Hamilton-Jacobi-Isaacs PDE. Our 
methodology, instead of formulating a general optimization 
problem for which the solution may not be easily com¬ 
putable, solves a special case exactly and efficiently. With 
this building block, we are able to solve more general 
problems through abstraction and refinement. 

Model predictive control (MPC) can as well be used to 
solve the reach-avoid problem [3], [24]. In each cycle of an 
MPC, the optimal input for reaching the goal while avoiding 
the obstacle, is computed for a fixed prediction horizon. 
Then, the first part of the optimal control input is applied, 
and a new input is computed from the new state, and so 
on. As the prediction horizon increases, the applied input 
converges to the optimal reach-avoid input. In the contrast, 
our approach can be used to synthesis controls for possibly 
unbounded horizon with safety and progress guarantees and 
can establish nonexistence of controller of certain type. 

There is a large body of results on automata theoretic 
approaches for controller synthesis [1], [16], [18], [23], [25]. 
The approach here is to construct a finite abstraction of 
the dynamical system and then invoke the LTL synthesis 
algorithms such as the one in [5]. This approach has been 
applied to several systems and several software tools for 
synthesis have been implemented [9], [19]. 

The authors of [17], [22] build Markov decision trees 
to synthesize control policies with maximum probability of 
satisfying the specifications. Our method is very different 
since we consider deterministic systems and try to synthesize 
controller that are guaranteed to satisfy the specifications. 

III. PRELIMINARIES AND BACKGROUND 

Sets and Functions: For a natural number N, [W] 
denotes the set {0,1,..., — 1}. Given two functions /, g : 

A K", we use d{f,g) = \f{a) — 5 (a) |oo to denote the 
£00 distance between / and g, where | • |oo is the standard 
oo-norm. 

We will use finite collections of sets to approximate 
arbitrary compact subsets in K". For a finite collection V 
of subsets of K" and a subset S c K", we say that V 
preserves S if there exists a subset V' ’A V such that (i) 
\J P = S, and (ii) V P e V\P', P n S = 0. In other 

PeV 

words, V completely and exactly represents S. 

A finite partition V of a compact subset S c K" is a finite 
disjoint collection of sets that exactly cover S. The resolution 
of a partition V is the maximum diameter of the sets in V. 
For two partitions of a compact set S, we say that V 

subsumes V, if for any I e V, there exists /' e V such that 
/ c I'. 

Piecewise Linear Systems, Feedback, and Robustness: 
A piecewise linear system M is a tuple {X,U,loc,I,P) 


where (a) X c K" is a compact set called the state space, 
(b) U c R™ is a compact set called the input space, (c) loc 
is a finite set called the set of locations, (d) I = {Pi}i(=ioc is 
a partition of X and each element of I is called a location 
invariant, and (e) P = {fi}ieioc is a collection of linear 
dynamic functions fi:XxU—>-X. 

The evolution of the continuous state of the system is 
governed by the dynamic function of the location invariant 
it is currently in. For any time f e N, a state Xt e P; and 
an input ut eU the next state of the system is given by the 
discrete-time dynamics: 

Xt+i = fi{xt,Ut). ( 2 ) 

A general static state-feedback control law can be thought 
of as a function u : X ^ U that maps each state to an input. 
In many systems, sensors and controller hardware have a 
finite resolution, and therefore, such a general law cannot be 
implemented. In this paper, we assume that M is associated 
with a controller table C which is a partition of the state 
space X and the u : C —> W maps each partition in C to an 
input. Essentially u is a look-up table, which assign an input 
for every equivalence class defined by C. 

For a feedback control policy u and a system (|^, the 
next state is just a function of the current state. We denote 
postM{x,\i) = fi{x,u{x)) if a; G P;. The subscript M is 
dropped if it is clear in the context. We denote by posfi{x, u) 
the state reached from x after the step. For a compact 
set of states S' c A, we define post{S,u) = {x' : 3a; G 
S such that a;' = post{x,u)}. The t step post operation 
posf* (S, u) is defined similarly. 

Our synthesis algorithm will be complete for system 
models upto some imprecision in the model. For a system 
M = {X,U, loc,I, P) and e > 0, another system M' is an 
e-perturbation of M if it is identical to M except that the 
set of dynamic functions for M' is P' = {f^ieioc, such that 
for each I G loc, d{fi, //) ^ e. We denote by Be{M) the set 
of all models that are e-perturbations of M. 

Reach-Avoid Control Problem (RA C): A reach-avoid 
control(RAC) problem is parameterized by the system 
model M, the controller table C, and three sets of states 
Init, Safe, Goal c X called the initial, safe and goal states. 
We will assume that these sets have some finite represen¬ 
tation (for example, hyperrectangles, polytopes). We define 
what it means to solve a RAC problem with a feedback 
control policy u. 

Definition 1. A solution to a RAC is a feedback control 
policy u : C —> such that for any initial state x G Init, the 
states visited by the system satisfies the condition: 

• (Safety) for all t eN, Xt G Safe and 

• (Progress) there exists T e N such that xt 6 Goal. 

Throughout the paper a RAC is uniquely specified by a 
model M as the rest parameters are fixed. 

IV. Constraint-based Synthesis 

A major barrier in encoding RAC as an SMT problem is 
that the safety and progress requirements are over unbounded 


time. Moreover, these requirements are stated in terms of 
the future reachable states of the system and computing 
that in and on itself is a hard problem. Instead of working 
with unbounded time reach sets, we address this problem 
by encoding a set of rules that inductively prove safety and 
progress of the control system. 

A. Inductive Synthesis Rules 

In addition to searching for the feedback control law u : 
C ^U, the SMT problem will encode the search for (a) an 
inductive invariant Inv ^ X that proves safety with u, and 
(b) a ranking function V that proves progress with u. 

In order to constrain the search, we will fix a template for 
the ranking function. For this paper, we will use the template 
C ^ N, that is, any function that is piecewise constant on 
the partition of the state space C. This choice has an easy 
interpretation: each entry in the controller table gives the 
rank of the controller along with the feedback law. Let V 
denote the countable set of all such functions. Each ranking 
function V eV maps every state in A’ to a natural number. 
For any C e C,V (C) is the natural number that all the states 
X e C map to. Now we are ready to present the basic rules 
encoding inductive synthesis of RAC: 


Find u 

: C ^ 14, V e V, Inv c X such that: 

Rl: 

Init ^ Inv 

R2: 

post{Inv,u) c Inv 

R3: 

Inv Safe 

R4: 

C c Goal ^ V{C) = 0 

R5: 

C c Inv A post{C, u) n C A 0 


^ F(C') > F(C") 

R6: 

C c Inv\Goal a post'^l^C, u) n C ¥= 0 


V(C) > ViC'). 


Fig. 1: Basic rules n(M, C, V) for synthesis for RAC. 


Rules R1-R3 imply that Inv is fixed-point of post with 
control u that contains Init and is contained in Safe, and 
therefore, is adequate for proving safety. Rule R4 states the 
the rank of any region C vanishes iff it is in Coal. Rule 
R5 encodes the (Lyapunov-like) property that the rank of 
any region C is nonincreasing along trajectories. Finally, 
rule R6 states that for any non-Goal region C, the rank 
decreases with u within k steps, where k is an induction 
parameter of this encoding. For RAC specified by model 
M with controller table C and a template V, we denote the 
SMT problem (Figure as n(M, C,V) For some control 
u, ranking function F e V and Inv c X, we write 
u,V,Inv ^ n(M, C, V)[^if the Rules R1-R6 are satisfied. 

Theorem 2 (Soundness). If u,V, Inv \= n(M, C,V), then 
u solves RAC specified by M. 

Proof. Let u, V, Inv satisfy rules in Figure Q. Fixing any 
X e Init, we prove safety and progress conditions separately. 

*For the sake of clarity, we supress the dependence on k. 


From Rl, x e Inv. Combined with R2, we have for any 
t e N, postu{x,t) e Inv. Since Inv c Safe (R3) we have 
postu{x, t) e Safe for any t. Thus the safety condition holds. 

We assume x e C such that Cn Goal = 0; otherwise the 
progress condition holds trivially. From R4 we have V(C) > 
0. From R5 and R6, in at most kV{C) steps, V decreases 
to 0. By i?4 this implies that x reaches the goal. □ 

a) Robustness Modulo Templates: In Section [III] we 
defined perturbations of system models, here we lift the 
definition to the corresponding synthesis rules: V') 

is an e-perturbation of n(M, C, V) if (i) the controller table 
and the ranking templates are identical C = C',V = V', and 
(ii) the model M' e Be{M) is an e-perturbation of M. 

Definition 3. For a controller table C and a template of 
ranking functions V, a RAC specified by M is robust modulo 
(C, V) if there exists e > 0 such that either of the following 
holds: 

(i) there exists a control u and a ranking function V e 

V such that for any M' e u,V,Inv \= 

n(M',C, V) with some Inv c X, or 

(ii) for none of M' G B^i^M), the synthesis problem 
n(M',C, V) is satisfiable. 

In Theorem]^ we will show that our synthesis algorithm 
is also relatively complete with respect to this notion of 
robustness. 

B. Weakened and Strengthened Rules 

The main challenge in solving n(M, C,V) is the post 
operator in Rules R2, R5, and R6. We need a reasonable 
representation of post for this computation to be effective. 
In this work, we use a finite partition V of the state 
space (which preserves C, Init, Safe, Goal) for computing 
the post. This choices is somewhat independent of the rest 
of the methodology and any other template (for example, 
linear functions, support functions, zonotopes) could be used 
instead of the fixed partitions. 

The key idea to solve it is to create over and under 
approximations of the post operator with respect to the 
representation of choice—in this case representation using 
the fixed partition V. These operators are then used to create 
weakened and strengthened versions of the basic inductive 
rules that can be effectively solved as SMT problems. 

We define an over-approximation (V-post) and an under¬ 
approximation (V-post) of the post operator with respect to 
a partition V as follows: for any compact S ^ X, 

V-pdst{S,u) = U 

PeV APr\post{S,\i}j^0 

V-post{S,n) = U 

PeV A PCpost (S,u) 

Roughly, the over-approximation 'P-post{S, u) computes the 
minimum superset of S which is preserved by V and 
the under-approximation 'P-post{S,u) computes the max¬ 
imum subset of S which is preserved by V. We define 










V-post* {S,u) and V-post*{S,u) as the t step over and 
under-approximations in the usual way. 

Proposition 4. For any measurable S ^ X, a post operator 
and any partition V, the following properties hold: 

(i) V-post{S, u) c post{S, u) c 'P-post{S, u), 

(ii) IfV subsumes V, then V' -post(S,u) 3 V-post{S^u) 
and V'-post{S,u) c 'P-post{S,n), and 

(Hi) For any e > 0, 3 (5 > 0 such that for any V with reso¬ 
lution less than 6, d(V-post(^S,u),V-post{S,u)) < e. 

Instead of searching for an exact inductive invariant 
Inv, the weakened and strengthened versions of the syn¬ 
thesis rules presented below try to find under (Must) 
and over-approximations (May) of the invariant using the 
V-post{S,u) and 'P-post{S,u.) operators. 


Find u 

: C ~*U, V eV, Must c X such that 

Wl; 

Init c Must 

W2: 

V-post{Must, \i) c Must 

W3: 

Must c Safe 

W4; 

C c Goal ^ V{C) = 0 

W5; 

C c Must A C' c P-post{C, u) 


^ V{C) > V{C') 

W6: 

C c Must\Goal A C' c V-post^{C, u) 


^ V{C) > V{C') 

Fig. 2 

: Weakened rules IPp{M,C,V) for synthesis. 

Find u 

:C-^U,V eV, May c A” s.t.: 

SI: 

Init c May 

S2: 

V-post{May,u) c May 

S3: 

May c Safe 

S4: 

C c Goal ^ V{C) = 0 

S5: 

C c May A C' c P-post{C, u) 


^ V{C) > V{C') 

S6: 

C c May\Goal a C" c V-post^"{C, u) 


^ V{C) > V{C') 


Fig. 3; Strengthened rules IV?p{M,C,V) for synthesis. 


Lemma 5. For any V, the following hold: 

(i) i/u, Inv ^ n(M,C, V), then there exist Must c X 
such that u, V, Must ^ np(M, C, V); and 

(ii) ifu,V,May \= Il'^(^M,C,V), then exists Inv £ X 
such that u, V, Inv \= n(M, C, V). 

Proof. Suppose u, V, Inv ^ n(M, C, V). We will show that 
there exists a Must c X satisfying the weakened rules (Wl- 
W6). Fix a u. From Proposition Bl the operator V-post{-, u) 
is upper bounded by posf(-, 11 ]. Since post{-,u) has a 
fixed point Inv, the fixed point of 'P-post{S,u) exists. 
Let Must be the hxed point defined by W1-W2 and we 
have Must c Inv. It follows that Must c Inv c Safe 
and W3 holds. W4 is inherited from R4. For any C c 
Must A C" c 'P-post{S,u)l^C), we have May c Inv and 


V-post{S, u)(C') c postu{C). From R5, therefore, V(C) ^ 
y(C") and W5 holds. Similarly, V-post(^S,u)(^C, k) c 
postu{C, k), thus W 6 also holds. Therefore, u,V,Must \= 

The proof of second part is similar. □ 

The above lemma states that the weakening and strength¬ 
ening of the synthesis rules are sound. With the additional 
robustness condition, we can show that either the former is 
unsatisfiable or the latter is satisfiable. 

Lemma 6 . If a RAC specified by M is robust modulo C, V, 
then there exists a sufficiently fine partition V such that 
either (i) I\.lp(^M,C,V) is unsatisfiable or (ii) I1^{M,C,V) 
is satisfiable. 

Proof. We discuss the two cases in Definition In this 
prove post, V — post, V — post without a subscript denote 
the operator with respect to model M. 

Suppose there exists e > 0 such that some controller u 
and ranking function V e V solves all e-perturbations of 
n(M, C,V). That is, for each M' e B,,(^M), there exists a 
InvM', such that u,V, Iuvm' \= n(M'). We define Inv^ 
as the union of all such InvM'’s. Roughly, Inv^ is the set 
of states that can be visited for some e-perturbation of M 
with controller u. Since every InvM' satisfies R3, R5, R 6 
in Figure it can be shown that (i) Inv^^ c Safe, (ii) 
C c Inv, A post{C,u) nC A 0 V{C) > V{C'), 
and (iii) C c Inv,\Goal a post'^{C,u) n C' ¥= 0 
V{C) > V(C'). Also, any subset of Inv, also satisfy the 
above three formula. From Proposition]^ for sufficiently fine 
partition V, for any S X, di^P-post(^S,u),post(^S,u)) sg 
e. We will inductively prove that the May set with re¬ 
spect to this partition 7^ is a subset of Inv,. (i) Initially, 
Init c Inv,. (ii) For any set S c Inv,, and any state 
X e V-post{S,u), it suffice to prove x e Inv,. First, 
since d(V-post(^S,u),post(^S,\i)) ^ e. We can find a state 
x' e post{S,\i) c Inv, such that ||x — x'\\ ^ e. Since x' 
is in Inv,, it is reached by some model M' e B,{M) for 
the first time. We construct a model M" that is identical 
to M' elsewhere except that at state x' the dynamics is 
x' = postM"{.x,\i). It is easy to show that M" is a e- 
perturbation of M which visits x with controller u. Thus 
X e Inv,. By (i) and (ii) above, we derive May c Inv,. 
It follows that the strengthened rules S3, S5 and S 6 are 
satisfied. In addition, S1-S2 is satisfied by the definition 
of Must and S4 is just inherited from R4. Therefore, the 
strengthened rules are satisfiable. 

Otherwise suppose exists e > 0 such that none of the e- 
perturbation of Il{M,C, V) is satisfiable. Again from Propo¬ 
sition 1^ for sufficiently fine partition V, for any S ^ X, 
d{'P-post{S, u),post{S, u)) ^ e. We on the contrary assume 
there exists some controller u,V,Must ^ Wfl^M,C,V). 
We define a model M' such that for any cell C e C 
and each states e C, the dynamics of M' is captured 
hy post = Proj{post{x,u),V-post{C,u)). The 
operator Proj{x,A) is a projection that maps a; to a 
state in A that is closest to x. It can be shown that 




































M' is an e-perturbation of M. Moreover, for any cell 
C X, postM'C = V-post{C,u). Thus, the problem of 
n(M",C, V) and are identical. It follows that 

u,V,Must \= n(M',C, V), which contradicts the fact none 
of e-perturbation of 11 is satisfiable modulo C,V. 

□ 


V. SMT-based Synthesis Algorithm 

We introduce an algorithm for controller synthesis for 
RAC using the strengthened and weakened inductive SMT 
encodings of the previous section. The algorithm takes as 
input the model M, the controller table/partition C, the tem¬ 
plate for the ranking function V and the three sets Init, Safe 
and Goal that define RAC problem. It iteratively refines the 
partition V for representing invariants and makes subroutine 
calls to the SMT solver with the strengthened and weakened 
rules until it either finds a controller law u or outputs _L. 
Specifically, in each iteration, (a) if the strengthened problem 
n|, is satisfiable then it returns the satisfying u. (b) if the 
weakened problem lip is unsatisfiable then it returns _L. 
Otherwise, (c) it refines the partition V (using the Must set 
computed from lip). The Refine{V, Must) function creates 


Algorithm 1: SMT-based Synthesis Algorithm 

1 input: M,C, V, Init, Safe, Goaf 

2 P <— init Partition', 

3 while True do 

4 {vals,u) Solve{nf,{M,C,V))', 

5 {valw, Must)Solve{n^{M,C,V))', 

6 if vals = SAT then 

7 I return u 

8 else if valw = UNSAT then 

9 I return _L 

10 else 

11 I V Refine{V, Must)', 

12 end 

13 end 


a finer partition of V. For the completeness result, we require 
that for any V, by iteratively applying Refine, the resolution 
of the resulting partition can be made arbitrarily fine. In 
Section [V-B| we discuss several heuristics for refinement that 
potentially improve the performance of the algorithm. 


In addition, we show that the algorithm is relative com¬ 
plete. That is, if RAC is robust modulo C,V, then the 
algorithm terminates with one of the above answers. 

Theorem 7. The algorithm is sound and relatively complete. 

Proof. Soundness. If the algorithm terminates and return u, 
then for some partition V, the SMT solver returns a satisfying 
solution u with Ilf,{M). From Lemma|^ u solves the RAC. 
Otherwise if the algorithm terminates and returns _L, then 
for some partition V, the SMT solver on Il'!p{M) returns 
UNSAT. From Lemma there is no control that solves the 
RAC problem modulo V. 

Relative Completeness. Since n(M) is a robust RAC 
modulo V, from Lemma we know that for a sufficiently 
fine partition, either np(M) is unsatisfiable or np(M) is 
are satisfiable. Thus the while-loop will terminate as the 
algorithm creates fine enough partitions. 

□ 


B. Guided Refinement 

There are different ways in which the refinement of the 
partition V can be implemented without compromising the 
soundness and the relative completeness guarantees. The 
naive strategy of subdividing every equivalence class in 
V, increases the size of the SMT problems quickly. As 
our algorithm solves both the weakened and strengthened 
versions of the problem simultaneously, we can marshall 
extra information in performing refinement. For example, 
when the weakened rules return a possible control u along 
with its proof V, Must, even though this controller u cannot 
be proven (to be safe and progress making) with the strength¬ 
ened rules, it can provide useful information for guiding the 
refinement. 

Definition 8. For a partition V and a set S that is preserved 
by V, V is a S'-guided refinement of V if V' is derived by 
refining the cells of V that are in S. 

One key observation is that, a X\Must-gmded refinement 
helps in generating safety proofs (S3 and W3), while a Must- 
guided refinement can improve the precesion of progress 
proofs (S5-S6 and W5-W6). The following proposition for¬ 
malizes part of this intuition and states that for given a 
controller u, refining the cells in Must does not improve the 
precision of the fixed-point Must, May computed by rules 
S1-S2 and W1-W2. 


A. Soundness and Relative Completeness 

We will next sketch the arguments for the correctness 
of the algorithm. Soundness of the algorithm implies that 
whenever it outputs u, (i) that u is a control law that solves 
the RAC problem, (ii) the May set obtained from solving 
Up in the final iteration is an inductive proof certificates for 
safety with u, and (iii) the U is a A:-step inductive proof 
certificate for progress with u. And, whenever the algorithm 
outputs L then there does not exists a controller u, a ranking 
function V e V and an invariant Inv such that the above 
(i)-(iii) holds. 


Proposition 9. For any control u, any set Init and any 
partitions P, let Must, May be the fixed point of operator 
P-po.stf,u) and P-post{-,u) containing Init. Let P' be a 
Must-guided refinement ofP' and Must', May' be the fixed 
point of P'-postf,\i) and P'-postf,u) containing Init. 
Then, Must = Must' and May = May'. 

By above proposition, a Must-guided refinement provides 
no help in generating better safety proofs. However, from 
Proposition a finer partition P increase the precision of 
P-post{C,u) and P-post{C,u). Since the rules S5-S6 and 
W5-W6 involve computing P-post{C, u) and P-post{C, u) 

















for cells in May and Must respectively, a Mttsf-guided 
refinement possibly increases the precision of these rules. 
Based on the above observations, we can adopt to the follow¬ 
ing heuristics for refinement; If the Must set is close to the 
unsafe set, perform X\Must-gmded refinement, otherwise 
perform Must-guided refinement. 

VI. Prototype Implementation and Experiments 

We implemented the synthesis algorithm in Python using 
the the CVC4 SMT solver [2]. In this section, we briefly 
report preliminary results on applying it to a simple class 
of navigation problems. With this implementation, we were 
able to automatically synthesize correct controls (and their 
inductive proofs) for some configurations and proved impos¬ 
sibility for others. 

Vehicle Navigation Problem: We consider a reach-avoid 
problem for a vehicle that follows piecewise linear approxi¬ 
mation of Dubin’s dynamics. The system model has 4 state 
variables [cc, y, u, 0]^: position, velocity and heading angle of 
the vehicle. It has input variables [a,/3]^: the acceleration 
and the turning rate. From the continuous Dubin’s vehicle 
model: x = vcosO, y = vsinO, v = a, 6 = u/3. we 
construct a switched linear model by partitioning the domain 
of 9 and v into 24 locations, and for each location we 
compute an approximate linear dynamics. The result is a 
switched-linear model: 

x'^ = x + av + b, = y+cv+d, = v + a, = 9 + e/3, 

(5) 

where a, b, c, d, e have different values in different locations. 
The piecewise linearized model preserves some properties of 
the original system. For example, the linearized model cannot 
turn in place: if the velocity is close to 0, the heading 6 
cannot change. Moreover, the velocity is non-negative, which 
further restricts its maneuverability. These properties give 
rise to interesting RA C problem instances where the system 
has no satisfying control law. 

We allow finitely many discrete input values and compute 
'P-post{C,u.) and 'P-post{C,u) offline as follows; For a 
given partition V, and a cell C e V, we first identify a set 
of cells M{C) such that C e M{C) can visit some state in 
C in one step. Then, for each possible input u, we compute 
the one step reach set of post(N{C),\i) with the help from 
reachability tools such as [12], [14]. Thus we just need to 
identify the input combinations such that C is covered by or 
intersected with post{Af{C), u). 

Experimental Results: We performed several experi¬ 
ments for the above class of problems using our prototype 
implementation. We search for a control policy as a look¬ 
up table, specified by a controller table. We utilize a 
controller table C with 768 cells in total. In Figure and 
the grids illustrate the projection of controller table to x, y 
coordinates. 

We create a partition V by further partitioning each cell in 
C into 4 pieces, with which we construct both the weakened 
and the strengthened rules. For some cases, we proved 
the impossibility of synthesis. We visualize such a case in 



Fig. 4; A RAC instance that is impossible to solve. The grid 
illustrates the controller table, the green block at the bottom 
left comer is Init, the blue rectangle at the top right is Goal, 
the smaller red blocks are unsafe. 
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Fig. 5; A RAC instance that has a satisfying control law. 
The lighter connected region is the Must set and the darker 
region together with the lighter region is the May set. 

Figure While for other cases, we successfully synthesized 
a control policy. An example is illustrated in Figure The 
satisfying control policy is synthesized with an inductive 
proof, namely the May set and the ranking function V. 

In the constraints of this synthesis problem, there are 
768 real-valued variables for control input in each cell, 
3072 integer variables for values of the ranking function 
for each partition and 3072 boolean variables indicates 
whether a partition is reached. The weakened or strengthened 
inductive rales are encoded in roughly 7000 constraints. The 
constraints are solved by CVC4 [2] in 10 minutes. 

VII. Conclusion 

In this work, we studied the controller synthesis problem 
of discrete-time systems with possibly unbounded time safety 
and progress specifications. Feveraging the growing strength 
of modern SMT tools, we propose an algorithm that finds 
controllers as well as inductive proofs of their correctness. 
Specifically, the algorithm creates a weaker and a stronger 
version of the synthesis problem and encodes them as SMT 
problems. By solving the controller synthesis problems for 
these two bounding systems automatically with SMT solvers, 
we can solve the synthesis problem for the original system. 
We prove that this algorithm is sound and relatively complete 
and show that the solution given by the strengthened system 





















































provide a guidance for refining the bounding system. Our 
experimental results based on a prototype implementation 
suggest that this can be a promising direction of investigation 
for controller synthesis research. 

Since the core problem of computing over-approximations 
of post are decoupled from synthesis in this formulation, 
one future direction of research that this work opens up is 
to extend this framework to nonlinear system models. The 
performance of the algorithm depends on the templates of 
the control, ranking function and invariants. Thus, to explore 
different classes of templates and study their performance in 
our synthesis framework is also a natural next step. 
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